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Abstract 

This paper introduces Manhattan sampling in two and higher dimensions, and proves sampling 
theorems. In two dimensions, Manhattan sampling, which takes samples densely along a Manhattan grid 
of lines, can be viewed as sampling on the union of two rectangular lattices, one dense horizontally, 
the other vertically, with the coarse spacing of each being a multiple of the fine spacing of the other. 
The sampling theorem shows that images bandlimited to the union of the Nyquist regions of the two 
rectangular lattices can be recovered from their Manhattan samples, and an efficient procedure for doing 
so is given. Such recovery is possible even though there is overlap among the spectral replicas induced 
by Manhattan sampling. 

In three and higher dimensions, there are many possible configurations for Manhattan sampling, 
each consisting of the union of special rectangular lattices called bi-step lattices. This paper identifies 
them, proves a sampling theorem showing that images bandlimited to the union of the Nyquist regions 
of the bi-step rectangular lattices are recoverable from Manhattan samples, and presents an efficient 
onion-peeling procedure for doing so. Furthermore, it develops a special representation for the bi-step 
lattices and an algebra with nice properties. It is also shown that the set of reconstructable images is 
maximal in the Landau sense. 

While most of the paper deals with continuous-space images, Manhattan sampling of discrete-space 
images is also considered, for infinite, as well as finite, support images. 

Index Terms 

image sampling, lattice sampling, Landau sampling rate, nonuniform periodic sampling. 
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Fig. 1. (a) 2D Manhattan-grid sampling sites with parameters fci = = 5 and Ai = A 2 . (b) Square lattice sampling at the 

same density, (c) Cross-shaped frequency support (centered at the origin) of images recoverable with Manhattan sampling. 


I. Introduction 

In the two-dimensional (2D) setting, Manhattan sampling (or M-sampling for short) is a recently 
proposed form of image sampling in which data is taken along evenly spaced rows and columns; the 
set of sample locations will be called a Manhattan grid. In particular, as illustrated in Fig. 1(a), given 
sampling intervals Ai, A2 > 0 and integers k\. kg > 1, samples arc taken at intervals of Ai along horizontal 
rows spaced & 2 A 2 apart, and also at intervals of A 2 along vertical columns spaced k ] A 1 apart. 

Manhattan sampling has been used to good effect in both lossy and lossless bilevel image compression 
[1]—[3]. These methods losslessly compress the samples in a Manhattan grid, for example with arithmetic 
coding (AC), which can be done with very few bits per M-sample because the samples are closely spaced 
and, hence, highly correlated. For lossless compression, the other pixels are then AC encoded, conditioned 
on those in the Manhattan grid, while for lossy compression there is no further encoding, and the decoder 
estimates the remaining pixels from those in the Manhattan grid. Markov random field models have been 
used to guide both the arithmetic coding and the estimation. 

M-sampling has also been proposed [4], [5], [7], [8] as a new approach to sampling grayscale images 
and other two-dimensional fields, with the motivations that (a) dense sampling along lines might capture 
edge transitions more completely than conventional lattice sampling with the same density, (b) sensor 
networks with a Manhattan deployment geometry need less power or less wire to transmit data than 
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conventional lattice or random deployments at the same density [7], [8], and (c) there are physical 
scenarios for which M-sampling is far more natural than traditional lattice sampling, such as when 
sampling from a moving vehicle, e.g., a ship sampling oxygen levels in a body of water. Similarly 
motivated by sampling from vehicles, the recent related work of Unnikrishnan and Vetterli [9], [10] 
considers sampling continuously along a grid of lines, i.e., with asymptotically large sampling rate. 

Methods for approximately reconstructing typical (non-bandlimited) images from M-samples have been 
developed in [4]—[6]. The present paper focuses on identifying a bandlimited set of images that can be 
perfectly reconstructed, as well as efficient methods for doing so. 

Manhattan sampling with parameters A i, A-2 - k\ , ko can be viewed as sampling on the union of the 
horizontally dense rectangular lattice consisting of all locations of the form (mAi, 772^2-^2), where 
ni,n2 are arbitrary integers, and the similarly defined vertically dense rectangular lattice consisting 
of all locations of the form (?zi Ati Ai , 712 A2). For brevity, we call these the horizontal and vertical lattices, 
respectively. 

By the conventional 2D sampling theorem [11] (see also [12, p. 72], [13, Chap. 3], [14, p. 43]), the 
samples on the horizontal lattice arc sufficient to distinguish and reconstruct any image bandlimited to the 
Nyquist region { (u, v ) : u < |u| < ok ] X2 }■ Likewise the samples on the vertical lattice arc sufficient 

to distinguish and reconstruct any image bandlimited to the Nyquist region {(it, v) : |it| < , | v | < 

2^}. Each of these samplings is maximally efficient in the Landau sense [15] that their sampling densities 
are as small as the area of the Nyquist region. Equivalently, the set of images bandlimited to the Nyquist 
region is maximal for the given sampling scheme. 

The first result of the present paper is a sampling theorem in Section III showing that images ban¬ 
dlimited to the union of these two Nyquist regions can be reconstructed from their samples on the 
union of the two rectangular lattices, i.e., on the Manhattan grid, and an efficient procedure for doing 
so is given. It is also shown that the images bandlimited in this way form a maximal reconstructable 
set for the Manhattan grid samples. As illustrated in Fig. 1(c), the union of the two Nyquist regions is 
the cross-shaped Manhattan region. We say that images whose spectra are confined to such a region 
are Manhattan-bandlimited. Given the relevance of Manhattan-bandlimiting, a figure in Section III will 
display the effect of several instances of such on a typical image. 

The principal goals of the remainder of the paper are to formulate M-sampling in three and higher 
dimensions, and to derive a sampling theorem and a reconstruction procedure. M-sampling in three 
dimensions can be motivated by the need to spatially sample a three-dimensional volume with a vehicle, 
or to spatio-temporally sample a two-dimensional region, as in video, or a spatio-temporal sensor network. 
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Four-dimensional sampling can be motivated by the need for spatio-temporal sampling of a three- 
dimensional spatial region. 

In three and higher dimensions, M-sampling can take a variety of forms. In order to describe two of 
these in three dimensions, consider the partition of 3D space into k\X\ x & 2 A 2 x /;•;>A 3 orthotopes (3D 
rectangles). As illustrated in Fig. 4(a), one form of M-sampling takes samples uniformly along each edge 
of each of these orthotopes — with spacing A* along edges parallel to axis i. Another form (Fig. 4(c)) 
takes samples uniformly on each face of each orthotope — with the samples on the face orthogonal to 
axis i taken according to a A j x A& rectangular lattice, where j and k denote the other dimensions. In other 
words, the first form samples densely along lines and the second samples densely along hyperplanes. 
Neither of these takes samples in the interior of any of the aforementioned orthotopes. 

More generally, as described in Section IV, M-sampling in an arbitrary dimension d is defined as 
taking samples on the union of some collection of d-dimensional bi-step lattices, which arc rectangular 
lattices defined by step sizes that in dimension i arc restricted to A, or /r ; A t . Thus, there are many possible 
M-samplings in d dimensions, even when A ? ’s and ki s arc fixed. We call such unions of d-dimensional 
bi-step lattices Manhattan sets. 

The main results of Sec. IV are (a) a sampling theorem showing that images bandlimited to the union of 
the Nyquist regions of the d-dimensional bi-step lattices comprising the Manhattan set can be distinguished 
by their M-samples, (b) efficient, onion-peeling procedures for perfectly reconstructing d-dimensional 
images, bandlimited as in (a), from their M-samples (one in frequency domain and one in spatial domain), 
and (c) a proof that the set of such bandlimited images is maximal in the Landau sense. 

The development of the sampling theorem and reconstruction procedures are enabled by an efficient 
parametrization of a bi-step lattice (with a given set of A,;’s and ki s) by a binary vector b = ( 61 ,... ,bd) 
indicating the dimensions i along which the spacing between lattice points is the smaller value, A,;, 
rather than the larger value, k t X t . This enables any Manhattan set to be compactly described by a finite 
set of b,’s (in addition to the A t ’s and ki s). A number of properties and relationships are enabled 
by this parametrization. For example, the computation of the density of a d-dimensional Manhattan 
set is enabled by a spatial partition whose 2 d atoms arc indexed by b’s. Similarly, the onion-peeling 
reconstruction procedures mentioned previously are keyed to a partition of frequency space whose 2 d 
atoms are indexed by b’s. The frequency-domain version reconstructs the image spectrum one atom at a 
time, beginning with “highest frequency” atoms (whose b’s contain the most l’s), and working towards 
the lower frequency atoms (whose b’s contain fewer ones). 
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In particular, as will be shown, the spectrum X b (u) in the atom indexed by b is computed via 

X b (u) = X b (u) - J>Y b '(u), (1) 

b': 11b'11>11b11 

where Xb(u) is the spectrum of the image samples in the bi-step lattice parametrized by b (a subset of 
the Manhattan set), the sum is over all b' with more ones than b, and AT b (u) is the spectrum of the 
samples (taken with the same bi-step lattice) of the image component x b (t) corresponding to atom b', 
which has previously been reconstructed. 

A discrete-space version of this requires only DFTs of the subsampling of the Manhattan samples 
and the previously reconstructed image components specified in the above, as well as summing and 
subtracting. Then an inverse DFT computes the newly reconstructed component. Summing all such 
components yields the reconstructed image. 

The method characterized by (1), and the discrete-space version thereof, can also be carried out in 
the spatial domain by applying the right-hand side of (1) to the corresponding sampled images, rather 
than their spectra, and then applying an ideal bandpass filter that extracts just the frequency component 
corresponding to atom b. The impulse responses of these filters will be given later. As will be seen, 
these impulse responses depend on the ki s and A*’s, but not the choice of bi-step lattices that comprise 
the Manhattan set. Moreover, the Aj’s have only a simple spatial scaling effect on the filters. 

Finally, we note that the development for three dimensions benefits greatly from the efficient parametriza- 
tion of bi-step lattices mentioned earlier, and that with such, it is possible to derive the M-sampling 
theorem and reconstruction procedure in arbitrary dimensions with essentially no additional effort or 
notation. 

We conclude the introduction by relating the present work to previous work. Multidimensional sampling 
theorems, showing that images with certain spectral support regions can be reconstructed from certain 
samplings sets, appeared first for lattice sampling sets in Peterson and Middleton [11], then later for 
unions of shifted lattices, i.e., lattice cosets, [16]—[27], although they were not always described as such. 

The earliest work [11], [16] required the spectral support region and sampling set to be chosen so 
that the spectral replicas induced by sampling did not overlap, and consequently, reconstruction could 
proceed simply by lowpass filtering the sampled image. For example, the approach of [16] could be used 
to reconstruct images from M-samples. However, it would require the images to be bandlimited to the 
Nyquist region of the coarse (rectangular) lattice, which is the intersection (rather than union) of the 
bi-step lattices comprising the Manhattan set. 
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Nonoverlapping spectral replicas were not required in later work [17]—[27], and more complex recon¬ 
struction procedures were proposed. Though not specifically intended for images, a seminal contribution 
stimulating a number of advances in image sampling was the multichannel, generalized sampling intro¬ 
duced by Papoulis [28]. For example, Papoulis’ framework is broad enough to include all image sampling 
schemes based on lattices and unions of shifted lattices. 

One difference between the present work and much past work is that we focus on a particular sampling 
set, namely a Manhattan set, and seek a largest possible frequency region such that any image handlimited 
to such can be reconstructed from the samples. In contrast, much of the past work [17]—[19], [23], 
[24], [29] focused on a particular frequency support region and sought a smallest possible sampling 
set, constructed from lattices and shifts thereof, such that images handlimited to this region could be 
reconstructed from such sampling sets. Nevertheless, some of the latter approaches could be used to 
reverse engineer reconstruction procedures and/or spectral support regions for Manhattan sets, as we now 
discuss. 

One substantial line of past work applies to sampling sets that consist of a sublattice of some specified 
base lattice , together with some of its cosets, each of which is a shift of the sublattice by some base 
lattice point. In this case, the subsampling corresponding to each coset (including the sublattice itself) 
can be viewed as a channel in a Papoulis multichannel, generalized sampling scheme. Consequently, the 
method of [28] can be applied. This is the approach taken by Marks and Cheung [17]—[19]. Since a 
Manhattan set can be viewed as the union of what we earlier called the coarse (rectangular - ) lattice and 
some number of its cosets with respect to the dense (rectangular) lattice , which contains all points t 
such that for each i, its /th coordinate is an integer multiple of A*, the Papoulis-Marks-Cheung (PMC) 
approach can be applied to Manhattan sets. 

In particular, Marks and Cheung focused on images with a given spectral support region and an initial 
base sampling lattice such that the induced spectral replicas of this support region do not overlap. They 
then showed that cosets of some sublattice could be removed from the base lattice until the sampling 
density was minimal (in the Landau sense) or approached minimal. Their method involved (a) partitioning 
the Nyquist region of the initial base lattice into atoms the size and shape of the Nyquist region of the 
sublattice, (b) counting the number of atoms of this partition that are not overlapped by any spectral 
replica of the designated support region induced by the initial base sampling lattice, and (c) showing 
that this number of sublattice cosets can be removed from the initial base lattice due to their samples 
being linearly dependent on other samples. If the atoms of the partition are too coarse to closely match 
the set of frequencies not contained in any spectral support replica, then choosing a sparser sublattice 
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will enable a liner partitioning, resulting in a higher fraction of the base samples being removed, which 
allows the sampling rate to be reduced until it equals or approaches the Landau minimum. 

With hindsight, one can apply their approach to a Manhattan sampling set. For simplicity, consider 
a 2D case and assume k\ = k% = k. Suppose images are bandlimited to the cross-shaped Manhattan 
region, and let the initial base sampling lattice and the sublattice be the dense and coarse rectangular 
lattices mentioned earlier. In this case, there arc k 2 cosets of the sublattice (including itself). One can 
then see that in the partition of the Nyquist region of the base/dense lattice into atoms having the size 
and shape of the Nyquist region of the coarse lattice, the number of atoms that are not contained in any 
sampled spectra is k 2 — (2k — 1). Thus, it is possible to remove all but 2k — 1 cosets, which is precisely 
the number of Manhattan samples in one k x k fundamental cell of the coarse lattice. Unfortunately, the 
PMC approach does not determine which cosets can be removed, so it does not directly tell us if the 
Manhattan samples arc sufficient to recover an image. While it does provide a matrix invertibility test 
that one can apply in any particular case to see if the Manhattan samples are sufficient, it is not clear 
how to analytically establish that one can remove all but the Manhattan samples in all cases. It is also 
not clear how the PMC approach would have lead to the discovery that the union of the Nyquist regions 
of the bi-step lattices is a reconstructable spectra support region for Manhattan sampling, especially in 
dimensions three and above. However, once it is known that the Manhattan samples arc sufficient for 
the spectral support region found in the present paper, then the Papoulis approach will directly lead to a 
reconstruction algorithm. 

As both the PMC and onion-peeling approaches involve partitioning frequency space, it is interesting 
to note that in dimension d the PMC approach requires a partition into nf=i atoms, whereas the 
onion-peeling algorithm partitions into only 2 d atoms. The smaller size of the latter partition is due to 
its being closely tailored to the specific structure of Manhattan samples. 

Similarly, in another line of work, Faridani [20] derived a sampling theorem and reconstruction formula 
for unions of shifts of one lattice. Given a spectral support region, the reconstruction involves partitioning 
this region in a certain way and setting up and solving a sizable number of systems of linear - equations, 
assuming that the equations have a solution. Since a Manhattan set can be viewed as the union of shifts 
of a lattice (the coarse lattice) and since we know from the results of the present paper that it is possible 
to reconstruct M-sampled images bandlimited to the Manhattan region, one could presumably solve the 
resulting equations to obtain a reconstruction formula. While this is interesting, finding the partition and 
setting up the equations can be difficult, especially in high dimensions. Thus, as before, the onion-peeling 
approach proposed in this paper is more natural, intuitive and straightforward to implement. 


February 6, 2015 


DRAFT 



While the PMC and Faridani approaches could be used to derive a reconstruction method for any 
Manhattan set, in their basic form, they do not provide direct closed form reconstruction methods, as 
given for example in this paper. That is, given sets of ki s and bi-step lattices, they outline a procedure that 
could be followed in order to derive a reconstruction method. Then, when the ki s or bi-step lattices are 
changed, the procedure must be followed again, essentially from scratch 1 . In contrast, the reconstruction 
methods given in the paper arc closed form, requiring just step-by-step following of the reconstruction 
formulas, which depend explicitly on the A,;’s, ki s and bi-step lattices. While it is conceivable that 
with enough work this alternative approach could be made closed form, it would appear to take much 
additional work, especially to make it apply to arbitrary dimensions. 

Behmard [26] derived a sampling theorem and reconstruction formula for unions of shifts of more 
than one lattice, which includes M-sampling, as it is a more general setting than [20]. However, the com¬ 
patibility conditions required to apply this approach are not satisfied by M-sampling and the Manhattan 
spectral support region. 

Other work on sampling with unions of shifted lattices includes that of (a) Venkataramani and Bresler 
[23], [24], which considered unions of shifted lattices in one dimension, and (b) Unnikrishnan and Vetterli 
[27], which considered unions of shifted lattices in higher dimensions. The latter include M-sampling 
and a reconstruction procedure was proposed with si mi larities to our onion-peeling approach, but which 
requires the spectral support region to be convex, which rules out the Manhattan region. Indeed, one 
of their examples is a 2D Manhattan grid, from which images can be recovered provided their spectra 
are bandlimited to a circular subset of the Manhattan region. Consequently, a significantly smaller set of 
images is reconstructable with their procedure. 

Finally, we mention that Manhattan-bandlimited spectra have been found to arise naturally in dynamic 
medical imaging applications, including both time-varying tomography [29] and dynamic MRI [30]. For 
example. Rilling et. al. [30, Fig. 1] give carotid blood velocity mapping as an example of a dynamic 
MRI application where a cross-shaped spectrum appeal's. Moreover, such spectra arise when temporal 
variation is localized to a small spatial area relative to the rest of the body, such as beating heart. With this 
motivation, Willis and Bresler [29] derived a single sampling lattice such that the cross-shaped spectral 
replicas did not overlap and the sampling rate was close to the Landau lower bound. In contrast, our 
sampling theorem also shows perfect reconstruction is possible. However, we sample with more than one 
lattice, the spectral replicas overlap, and the Landau bound is met exactly. 

'The method can be derived assuming unit Ads and then spatially scaled for the actual Ads. 
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In summary, given that the present paper shows that images bandlimited to the union of the Nyquist 
regions of the bi-step lattices of a Manhattan sampling set can be perfectly reconstructed from the 
Manhattan samples, there arc probably a number of alternative ways to derive reconstruction algorithms. 
In the view of the authors, the onion-peeling method, whose development was guided by the specific 
structure of Manhattan samples, is a natural and efficient reconstruction method with a straightforward 
interpretation in frequency space. It is also closed form in terms of the parameters of the Manhattan set. 
In addition, the union-of-bi-step-lattice viewpoint taken in this paper leads naturally to the hypothesis 
that the union of Nyquist regions is a support region of images that are reconstructable from Manhattan 
samples. It is not known if other approaches would have lead investigators to this region. 

The paper is written so that the reader who is primarily interested in 2D images can focus on Sections 
II, III, and V. 


II. Preliminaries 

This section provides background and notation for sampling and lattices that will be used throughout 
the the paper. 

Let M denote the real numbers, let K' :/ denote d-dimensional Euclidean space, let Z denote the set of 
all integers, and let Ti d denote the set of all integer-valued d-dimensional vectors. In dimension d, an 
image is a mapping x(t) : W :l —> M, where the spatial variable is t = (t ±,..., tf). We restrict attention to 
images x(t) that contain no delta functions or other generalized functions, and have well defined Fourier 
transforms containing no delta functions or other generalized functions, where by Fourier transform we 
mean 

2 f(u) = d 7 {x(t)} = j x(t) e ~ j2nt u dt. 

We will often refer to X(u) as the spectrum of x(t). 

Sampling a d-dimensional image x(t) means collecting its values on some countable sampling set §. 
That is, it produces the set of values {x(t) : t E S}. As commonly done, one can model such sampling 
as multiplication of x(t) by the comb function of the set S, which produces the sampled image 

xs(t) = x(t )K§y2 d(t-t'), ( 2 ) 

t'e§ 

where A); is a normalizing constant and Aft) denotes the Dirac delta function in d-space. The Fourier 
transform of x§(t) is then called the sampled spectrum. 

Rectangular sampling refers to sampling with a rectangular lattice. Given d and a = (a\,... ,af) 
with positive components, the d-dimensional rectangular lattice with step vector a is a countably infinite 
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set of points that are spaced by integer multiples of the step size on in the ith dimension. Specifically, 

L(a) = {t : ti is a multiple of a*, i = 1,..., d} 

= {t = n © a : n G Z d } , 


where © denotes element-wise product (also known as the Hadamard or Schur product). Alternatively, 
L(cx.) is the additive group generated by the basis a:ic[,..., q,/g,/, where C|..... c,j is the standard basis, 

i.e., c,; has a 1 in the ith place and 0’s elsewhere. That is, 

d 

L(a ) = jt = n i a i e i '■ n £ j • 

i =1 

The reciprocal lattice coiTesponding to L(a) is 


L*(a ) = L(q-\ 


.., a 



When sampling with set S = L(a), it is convenient to set the normalizing constant in (2) to be 


Ks = 


( 3 ) 


With this, the sampled image, denoted a; a (t), has spectrum 

-X'ci(u) = x ( u ~v). (4) 

veL* (a) 

^From this, one sees that the sampled spectrum X a (u) consists of replicas of the original image spectrum 
X(u), translated to the sites in frequency domain of the reciprocal lattice. The usual (/-dimensional 
sampling theorem follows from the fact that if the support of X(u) lies entirely within the Nyquist 
region 2 

M a = ju : \ui\ < i = 1 ,..., dj , 

then said replicas do not overlap, and consequently, the original spectrum can be recovered by extracting 
the portion of the sampled image spectrum in the Nyquist region. 


III. Two-Dimensional Manhattan Sampling 

As introduced earlier and depicted in Fig. 1(a), Manhattan sampling (M-sampling) uses locations spaced 
closely along a grid of horizontal and vertical lines. In particular, we assume there is a sample at the 
origin, as well as samples spaced Ai apart on horizontal lines spaced AoA -2 apart, and samples spaced A 2 

Tn this paper, script variables such as JV, B or A will usually denote subsets of frequency space. 
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apart on vertical lines spaced k\X\ apart, where A t > 0 and k \, k -2 are integers greater than one 3 . The 
issue, now, is to find an as large as possible set of images that can be perfectly reconstructed from these 
samples, as well as an efficient procedure for doing so. 

A first thought is to model M-sampling as multiplying the given image at (t) by a comb function having 
delta functions at the Manhattan sampling locations, and then to analyze the spectra of the resulting 
sampled image. Since this comb function has the same periodicity as a comb function for the coarse 
lattice Lc = A (A:i A i, & 2 A 2 ), the replicas of the image spectrum lie at frequency sites in the reciprocal 
lattice L* c , or a subset thereof. Thus, perfect reconstruction is possible for images bandlimited to the 
Nyquist region Me of the coarse lattice Lc- However, since such reconstructions need only use samples 
in the coarse lattice, it may be that a larger set of images is reconstructable from the full Manhattan grid. 
On the other hand, if images are bandlimited to a region larger than Me, c.g., a scaling of the Nyquist 
region such as {l + e)Mc, e > 0, then the spectral replicas induced by an M-sampling comb may overlap. 
Even if this does not eliminate the possibility of perfect reconstruction, it will at least complicate the 
analysis. 

Accordingly, we pursue an approach that does not rely on nonoverlapping replicas, but derives from 
the key observation that the Manhattan sampling set can be viewed as the union of two rectangular 
lattices. Let us initially focus on what can be recovered from the samples of each lattice by itself. The 
horizontal lattice, Lh — L(Ai, A 9 A 2 ), densely samples in the horizontal direction and coarsely samples in 
the vertical direction; the vertical lattice, Ly = IXk\X\. A2), coarsely samples in the horizontal direction 
and densely samples in the vertical direction; and the sampling set for M-sampling is 

_M(A;k) = Lj[ U Ly . 

Note also that the intersection of the two lattices is the coarse lattice Lc, whose comb function has the 
same periodicity as a comb function for the Manhattan grid. 

Clearly, all images bandlimited to the Nyquist region Mh of the horizontal lattice Ly can be recovered 
from just the samples in this lattice. Likewise, all images bandlimited to the Nyquist region My of the 
vertical lattice Ly can be recovered from just the samples in this lattice. Each of these by itself leads 
to a larger recoverable set of images than the set recoverable from sampling with the coarse lattice Lc- 
However, neither type of sampling and reconstruction uses all of the M-samples. 

We now show how images bandlimited to the union of the Nyquist regions of the horizontal and 
vertical lattices can be recovered from the full set of M-samples. Specifically, suppose image x(t) is 

3 We require k\,k ,2 > 1 since if fci = 1 or fe = 1. the sampling set reduces to a normal rectangular lattice. 
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(a) 




Fig. 2. For Manhattan sampling with Ai = At and fci = fe = 3: (a) Support of the sampled spectrum for an image bandlimited 
to the Manhattan region, when sampled with the fine lattice L\ lt \ 2 . The original spectrum is black with a white x in its center, 
whereas replicas are white with a black x in their centers, (b) Support of the sampled spectrum when sampled with the vertical 
lattice Lk 1 x 1 ,\ 2 - Gray indicates regions where replicas overlap either the original spectrum or each other, (c) Same as (b), except 
that the sampling is with the horizontal lattice L\ lt k 2 \ 2 ■ 


bandlimited to AT(A; k) = Nh uNy, which is the cross-shaped region shown in Fig. 1(c). First, consider 
only the samples of x(t) taken on the vertical lattice. Since the cross-shaped region AT (A: k) is not 
contained in the Nyquist region A fy, the replicas of X(u) may overlap in the spectrum of the vertically 
sampled image, as illustrated in Fig. 2(b), However, certain portions of each cross-shaped replica cannot 
be overlapped, and thus these portions of the spectra of x(t) can be immediately recovered. 

Specifically, it is easy to see that with vertical sampling, the vertical highpass region By = Ny — Nc 
is not overlapped. Thus, with /d(ii) denoting the indicator function of some set D and A' (u) = 
X(u)/g v (u) denoting the portion of X(u) in By, one sees that from the vertical samples and their 
spectrum Xy(u), one can immediately recover via A' (u) = Xy(u)Iy v (u). Likewise from the 

horizontal samples and their spectrum Xh{ u), the horizontal highpass region By = My — Nc is not 
overlapped. Thus, one can immediately recover X H {u) = X(u)Iq h (u) = Xy(u)Iy n ( i.i ). 4 

Since Af l/ (u) and X H (u) are now known, and X(u) is bandlimited to AT (A, k) = By U By L)Nc, it 
remains only to find X c (u) = Af(u)/y c (u). It will then follow that AT(u) = 2f' / (u) + 2f // (u) + 2f c ’(u). 
Inverse transforms will give x(t) = N (t) + x H (t) + ^^(t). 

To determine X c (u), consider the vertical sampling of x(t), and observe in Fig. 2(b) that the overlap 
of the image spectrum AT(u) in Nc by the various spectral replicas is due only to replications of the 

throughout the paper, a superscript on an image x or spectrum X will usually pertain to a frequency region, and a subscript 
will usually pertain to a sampling. 
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horizontal highpass frequency components in Bh■ Since these have already been determined, it ought to 
be possible subtract their effects. 

To see that this can be done, let us focus on Xy(u) //v c (u). ,-From (4) and the fact that X(u) = 0 
for u 0 A4(A, k), we have 


X v (u)I Mc (u) = ]T X 


-OVc(u) , 


where n = |_^J. Now using 2T(u) = X v (u ) + X H { u) + X c (u) in the above along with the facts that 

(a) X v (ui - k^,u 2 ) /,v c ( u ) = 0 for all i, 

(b) X H (ui ~ kj^,u 2 ) Im c ( u ) = 0 for * = 

(c) X c (ui - ^ ,u 2 ) IM c ( u ) = 0 unless i = 0, 
we find 


where 


X v (u)Im c (u) = X c (u) + Y(u)/y c (u), 


Y(u) 4 


n 

“ kM ' U2 ) Ie ~( Ul " GaT'“ 2 )' 


where the last equality uses the fact, mentioned earlier, that X H (u) = Xh(vl)Iq h (u). Notice that Y (u) 
is the component of Xy(u) due to aliasing by replicas of X H (u), and is directly computable from the 
horizontal samples. It follows from (5) that X c {\i) = (Xy(u) — Y(u)) Im c ( u ). 

In summary, a procedure for recovering a cross bandlimited x from its M-samples is 

1) Compute the spectra, Xjj{\i) and Ty(u), of the horizontally and vertically dense samples, respec¬ 
tively. 

2) From Xh{ u), compute Y(u) for u G Me- 


3) Let 


Xy(u), 
^( u ) = Uff(u), 


U G By 
u G B H 


Xy(u) - Y (u), U G Me 
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4) Let x(t) be the inverse Fourier transform of X{\\). 

This result is summarized in the following. 

Theorem 1: 2D Manhattan sampling theorem. Given Ai, A 2 > 0 and integers k\, k 2 greater than 1, any 
image x(t) whose Fourier transform is bandlimited to the cross-shaped region Xt(\: k) can be recovered 
from its M-samples in M (A; k) with the procedure given above. 

The following alternative expression for Xy(u) /y c (u) will lead to an easier to implement procedure 
for discrete-space images with finite support (presented later). Using (4) and X(u) = X y (u)+X^(u) + 
X c (u), we find 

AV(u)/v c (u) = ^2(x Y (u-v) + X H ( y u-v) + X c (u-v)^Ijv c (u) 

v£Ly 

= X c (u) + J2 xH ( u ~ v ) u ) 
veiF 

= X c (u) + Y'(u)Im c (u), 

where 

y'(u) ^ ^X^(u-v). (8) 

veL^ 

It follows that y(u) in the procedure given previously can be replaced by y'(u). The advantage is that, 
as shown below, Y'(u) can be computed with Fourier transforms instead of a summation. To show this, 
let Sy denote the vertical sampling operator, which when applied to an image z{t) produces ZL v (t ) as 
defined by (2). We recognize the summation in (8) as the sampled spectrum when the image x H (t) is 
vertically sampled. Since x H (t), and consequently X ,f (u), can be computed from the horizontal samples, 

y'(u) = x{s v {x H ( t)}} = 

While the above may initially appear - complex 5 , in the discrete-space, finite-support case discussed shortly, 
it will lead to a simple procedure that avoids the summations in (6) and (8). 

Maximality, in the Landau sense, of the set of reconstructable images 
The sampling density of an M (A; k) M-sampling set is 

_ k\ + k2 - 1 
^ k\k2\\\2 

5 Note also that the expression (8) for y'(u) contains more terms in the sunt than the corresponding expression (6) for y(u). 
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since any AqAi x ho A 9 rectangle in M 2 contains k\ + hp — 1 samples. In the frequency domain, the area 
of the Manhattan-bandlimited region, denoted yV! (A: k)|, is the sum of the areas of Bh, By and Me- 
Alternatively, it is sum of the areas of the Nyquist regions corresponding to the horizontal and vertical 
sampling lattices, minus the area of their intersection. Either way, this may be written as 


\Bh\ + \By\ + \Nc\ 


1 1 1 
&1A1A2 Ai 2 AiA 2 fcifc 2 AiA 2 


which simplifies to the previous expression for sampling density p. Thus, the set of images bandlimited 
to the Manhattan region Ad(A;k) is a maximal set of reconstructable images in the Landau sense for 
the M-sampling grid M(A;k). 


Discrete-space images 

In this section, we briefly consider M-sampling of discrete-space images. Such images might be created 
by rectangularly sampling a continuous-space image, or they might exist only as discrete-space objects. 
In any case, we consider an image to be a mapping x[t] : T —> M where T is either the (infinite) 
integer lattice Z 2 , or a finite subset of the form T = {t : 0 < t\ < T\ — 1, 0 < £2 < T 2 — 1} for 
some positive integers T\,T -2 . Sampling x[t] refers to collecting a subset of its values. In the infinite 
support case, T = Z 2 , a Manhattan grid M(A, k) is once again defined to be the union of a horizontal 
lattice Lh — L(X\, 1 ^ 2 X 2 ) and a vertical lattice Ly = L(k\X\, X 2 ), except that now each lattice must 
be a sublattice of the integer lattice Z rf , i.e., Ai,A 2 must be positive integers. In this case, we assume 
the discrete-space Fourier transform of x[t] is well defined and contains no delta functions or other 
generalized functions. In the finite support case, a Manhattan grid is formed in a similar way, namely, 
M (A, k) = L ff U Ly, where now Ly and Ly are truncated to the finite T. 

(a) Infinite-support discrete-space images: In this case, Theorem 1 holds with only trivial changes, as 
does the reconstruction procedure. Specifically, the only required changes are: (i) replace the continuous- 
space Fourier transform as the formula for a spectrum with the discrete-space Fourier transform, and (ii) 
scale all specified frequencies by 2-k, such as those defining Nyquist regions and Af(A;k). 

(b) Finite-support discrete-space images: In this case, as is customary, we use the Discrete Fourier 
Transform (DFT) as the formula for the spectrum of an image: 

X[u] = ^x[t]e _j27r( ^ tl+ ^ s) , ueT. 

teT 

The conventional sampling theorem (c.f. [26]) for discrete-space images with spatial support T (defined 
by Tj, T 2 ) sampled with a rectangular lattice L(o \ . 0 - 2 ) limited to T says that an image x[t] with support 
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T can be recovered from its samples in this lattice if T\ and T-> are integer multiples of a\ and a: 2 , 
respectively, and its DFT A[u] is zero outside the (discrete) Nyquist region 

Af au a 2 = {u € T : for i = 1 & 2, 0 < m < or T { - Z±_ < m < T t - 1) . 

v loti Zcx^ j 


Now suppose a finite-support discrete-space image x[t] is sampled on the Manhattan grid M(A, k) 
and is bandlimited to the cross-shaped Manhattan region 


M(X, k) = Mh U My 1 


where Mh and My arc the Nyquist regions of the horizontal and vertical lattices, respectively. Assuming 
T\ and T 2 are integer multiples of k ] A 1 and 7 2 A 2 , respectively, a straightforward adaptation of the analysis 
for continuous-space images shows that from the samples in the vertical lattice Ly, one can recover the 
spectrum X [u] in the highpass region By = My — Me, where My and Me are the Nyquist regions 
of the vertical and coarse lattices, respectively. Specifically, from the spectrum Ay[u] of the vertically 
sampled image xy[ t] (with scaling as in (2)-(3)), one recovers X v [u] = X\u\Ig [u] = Xy[u]fg [u]. 
Likewise, from the samples in the horizontal lattice Lh, one can recover the spectrum in the highpass 
region Bh — Mh —Me from the spectrum X//[u] of the horizontally sampled image xh [t] via X H [u] = 
X[u]/g (u) = XhM\I§ h [u]. Finally, the spectrum in the Nyquist region Me of the coarse lattice can 
be determined via A' r; [u] = (Ay [u] — Y [u]) 7^. [u], where 

Aii Ai — 1 

Y[u] = feiAiA 2 ^ X H 

r =1 

This leads to the following. 


u\ — r 


Ti \ 


—jmodTy^ 


Theorem 2: 2D discrete-space, finite-support Manhattan sampling theorem. If T\ and T 2 are integer 
multiples of k\ X\ and /;: 2 A 2 , respectively, then an image x[t] with finite support T can be recovered from 
its M-samples in M(A, k) if its DFT A[u] is zero outside the Manhattan region MKX. k). 

Reconstruction procedure: 

Given the samples in Manhattan grid M(X, k) of an image x[t] bandlimited to A4(X, k), the following 
adaptation of the continuous-space procedure recovers the entire x [t]. 

1) Let xy[ t] equal k\X \ A 2 x[t] on the vertical lattice Ly and zero otherwise, and let [t] equal 
A' 2 A 1 A 2 x[t] on the horizontal lattice L/r and zero otherwise. Compute Ay[u] = DFT{xy[t]} and 
X H [u] =DFT{x H [t]}. 

2) Compute the “alias subtraction” term 

Y'[u] = DFT{»SV{IDFT{ Ig H [u] DFT{x/r[t]}} } }, 
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Fig. 3. (a) Original 256 x 256 image, (b) Image bandlimited to Manhattan region M (A, k), with fci = fe = 4 and Ai = A 2 = 1. 
(c) Same as (b) except fci = kz = 8. (d) Same as (c) except Ai = A 2 = 2. (Note: after spectra were zeroed outside At (A, k), 
inverse transforms were applied, negligible imaginary parts were discarded, and images were quantized to {0,1,..., 255}.) (e) 
Image sampled with parameters of (c) and reconstructed without first bandlimiting to Manhattan region. Log magnitude spectra: 
(f) original image; (g) original image sampled with parameters of (c); (h) original image bandlimited with parameters of (c); (i) 
bandlimited image (c) sampled with parameters of (c); 


where Sy denotes the vertical sampling operator that, when applied to an image z\ t], produces an 
image that is &iAiA 2 z[t] on Ly, and zero elsewhere. 

3) Compute the spectrum: 


Xv[u], 

U £ By 

X H [ u], 

u E B H 

X v [u]-Y'[u], 

u £ Nc 


4) Invert the spectrum: 


x [t] = idft{a:[u]}. 


This reconstruction procedure uses 5 DFT/IDFT operations, each requiring 0(N log N) arithmetic 
operations when implemented with an FFT, where N = T 1 T 2 , plus three pairwise additions of T\ x T 2 
matrices, each requiring T 1 T 2 additions, plus instances of setting matrix elements to zero. In summary, 
the complexity of reconstruction, which is dominated by the FFT’s, is 0(N log N) operations per image. 

Note that few real world, finite-support images will satisfy the conditions of Theorem 2. As a result, 
to apply M-sampling to a real world image, the image can be pre-processed by zero-padding so that 
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its dimensions are multiples of k±\i and A 2 A 2 , respectively, and “Manhattan filtering” by taking the 
DFT and setting to zero all coefficients outside of M.(\, k). Such padded and filtered images can 
be recovered perfectly from their M-samples. To illustrate the effects of such filtering, which heavily 
suppresses diagonal frequencies, Fig. 3 shows a finite-support image and its filtering with several choices 
of parameters. It also shows the spectra of the image before and after bandlimiting, the spectrum of the 
sampled image, with and without bandlimiting, and the effect of sampling and reconstruction without first 
pre-filtering. Note that the image was chosen to have sharp edges surrounded by a smooth background 
in order that one can easily see the ringing due to bandlimiting. 


IV. Higher-Dimensional Manhattan Sampling 

A. Introduction 

As mentioned earlier, in any dimension d > 3 there are a number of possible d-dimensional Manhattan 
sets. Each is a finite union of rectangular lattices, each defined by step sizes that in dimension i arc 
constrained to be A,; or Ay A,;, where each A, is a positive constant called the dense spacing in dimension i, 
and each A x is an integer greater than 1 called the sampling factor in dimension i. Such a rectangular lattice 
will be called a (A, k)-lattice, where A = (Ai,... , X,i) is its dense spacing vector and k = (At, ... , Ay/) 
is its sampling factor vector. It will also be called a (d, A, k)-lattice when we wish to emphasize d, and a 
bi-step lattice when we do not wish to specify parameters. (The term “bi-step” emphasizes that each step 
size ay can only take one of two values: A/ or A, A,). Accordingly, to specify a d-dimensional Manhattan 
set , one specifies a dense spacing vector A, a sampling factor vector k, and a collection of (A, k)-lattices. 

To efficiently characterize a (A, k)-lattice, we let b = (b \,..., bf) denote a d-dimensional vector, called 
its bi-step indicator vector, or more concisely bi-step vector, that indicates the dimensions along which the 
bi-step lattice is dense, according to the convention bi = 1 if the step size is A, in dimension i and 0 if the 
step size is Ay A/. Thus, the (A, k)-lattice specified by b is T,\,k.b — -A( a b) ? with Qb = (at>,i, • • •, ctb 4 ) 
defined by 


or equivalently. 


,i 


{ A;, bi — 1 

AjAi, bi — 0 


( 9 ) 


^A.k.b — {t : tj is a multiple of A,A,; for i s.t. b, = 0, and a multiple of A* for other i } . (10) 
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Note that we generally consider d, A and k to be fixed, and so as an abbreviation and slight abuse of 
notation, we usually write L b instead of ^A,k,b- It will also be useful to let .x'h(t) and Xb(u) denote, 
respectively, the sampled image and the sampled spectrum due to sampling x(t) with L b- 

The following summarizes. 

Definition 1: Given dimension d, dense spacing vector A, sampling factor vector k (all of its com¬ 
ponents arc integers greater than 1), and a finite collection of id, A. kj-lattices specified by the bi-step 
vectors in B = {bi,... b m }, the corresponding id. A, k, B)-Manhattan (sampling) set is 

m 

M (d, A, k, B) ± |J L hj ■ (11) 

3 =i 

As d, A and k will be considered fixed, we usually write M ( B) instead of M(d. A. k. B). B will be 
called a Manhattan collection or M-collection for short. 

B. Examples and properties of bi-step lattices 

It is useful to call attention to certain bi-step lattices. One is the dense lattice L\ corresponding to 
the bi-step vector b = 1 = (1,..,, 1). It is a rectangular - lattice with step vector A that contains every 
other (A, k)-lattice. Another is the coarse lattice Lq corresponding to b = 0 = (0,..., 0), which is 
the rectangular - lattice with step vector a = k 0 A and which is contained in every other (A, k)-lattice. 
As mentioned in the introduction for 3D Manhattan sets, it will be useful to consider the partition of 
M. d induced by the coarse lattice, whose cells arc A;i A i x ... x Ay/A,/ orthotopes (hyper-rectangles) with 
corners at lattice points. These orthotopes will be called fundamental cells. The coarse lattice contains 
just the corners of these fundamental cells; other (A, k)-lattices may contain points on their edges and 
faces, but only the dense lattice L\ contains points in their interiors. 

A third lattice to consider is L e . corresponding to bi-step vector b = e,, which can be viewed as 
a collection of points spaced densely on lines parallel to e t , with one line passing through each point 
of (d — 1 (-dimensional cubic lattice L(k 1 A 1 ,..., fcj + iA/ + i,..., kd\d )• Finally, we mention the 

lattice corresponding to b = 1 — e,;, which can be viewed as sampling densely on shifts of the d — 1 
dimensional lattice L(Ai,..., Aj_i, Aj+i,..., Ad) spaced Ac./A* apart. 

Given two (binary) bi-step vectors bi and b 2 , define their union bi V b 2 and intersection bi A b 2 to 
be their element-wise ‘OR’ and ‘AND’, respectively, and define bi C b 2 to mean bi A b 2 = bi. Define 
the complement to be 1/ - 1 - b, and the Hamming weight or simply weight ||b|| to be the number of 
ones contained in b. 

The following are useful properties of 7 representations of bi-step lattices. 
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Fact 1: Considering (d, A, k)-lattices, 

(a) L\ 3l C Lb 2 if and only if bi C b 2 , 

(b) L\ 3l = Lb 2 if and only if bi = b 2 , 

(c) n Cb 2 -f / biAb 2 j 

(d) If Lg C u 7= ^ L\y ., then for some j , Lg A .| > - and consequently from (a), b CZ b^. 

Proof: 

(a) and (b) are elementary. 

(c) Ly )l Cl Lb 2 

= {t : ti is a multiple of k,X t for all i s.t. b\ A = 0 or lr> A = 0, and A, is a multiple of Aj for other i } 

= {t : t{ is a multiple of k,\ for all i s.t. (bi A 1)2),= 0 and A, is a multiple of A i for other if 

= -^biAbo 

(d) As is well known, for m = 2 this property derives from just the group nature of lattices, but not for 
larger values of m. Accordingly, to prove it for arbitrary m, we need to use properties of (A, k) lattices. 
Specifically, we demonstrate the contrapositive. Suppose (jL L^., j = 1 ,m. Then from (a), for 
each 1 < j < m, b (£_ b ; , and so there exists i ;] such that bi j = 1 and b hl/ = 0. Let / denote the set of 
all such if s, and let t = (ti,..., tf) be defined by 

[ (hi + l)Aj, if i £ I 
ti = < 

I kiXi, otherwise 

Refening to (10), we see that t <E Lg, because the only dimensions i for which A* is not a multiple of 
k t A,; arc those in /, in which case bi = 1, i.e., the lattice Lg is dense in dimension i. Moreover, again 
referring to (10), we see that for each j € {1,..., to}, t 0 L\ 3j , because A* = (k l;t + l)Aj is not a multiple 
of fcqAj, yet bj ) i j = 0, i.e., the lattice Lb, is coarse in dimension ij. It follows that t f UT =1 Lb r Hence, 
L~ h <tUf = 1 L h] . □ 

Among other things, (b) shows there is a one-to-one correspondence between bi-step vectors b and 
{d. A, k) lattices, which verifies that the b’s are valid representations of the (A, k)-lattices. Also, since 
there are 2 d possible bi-step vectors b, it follows that there arc 2 d distinct (d, A, k)-lattices. 

Note that while (c) shows that the intersection of two (d. A, k)-lattices is another id. A, k)-lattice, such 
is not true for the union, which is why a Manhattan set is not ordinarily a lattice. For example, when 
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(a) (b) (c) (d) 

Fig. 4. Examples of 3D Manhattan sampling M(B) and their corresponding Manhattan regions M(B). (a) Manhat¬ 
tan lines B = {(1,0,0), (0,1,0), (0,0,1)}, and (b) its corresponding Manhattan region, (c) Manhattan facets B = 
{(1,1, 0), (1, 0,1), (0,1,1)}, and (d) its corresponding Manhattan region. 


d = 2, L ei U L e2 is the Manhattan set M({ei,e 2 }) shown in Fig. 1(a), which is not a lattice and does 
not equal L ei ve 2 , which is the dense lattice Ln^y 

C. Examples of Manhattan sets 

Several types of Manhattan sets deserve special attention. 

1) Manhattan lines is a Manhattan set M(B) specified by B = {ei,..., e^}. In this case the samples 
are taken on the ID edges of the fundamental cells, i.e., on d orthogonal sets of parallel lines in 
W 1 . See Figures 1(a) and 4(a) for illustrations of Manhattan lines in two and three dimensions, 
respectively. 

2) Manhattan facets is a Manhattan set M(B) specified by B = {ef,..., e^}. Sampling on a set of 
Manhattan facets is analogous to sampling densely along d orthogonal sets of parallel hyperplanes 
in R d . See Figures 1(a) and 4(c) for illustrations of Manhattan facets in two and three dimensions, 
respectively. For d = 2, Manhattan facets and lines are identical. 

3) Though technically any (A. k)-lattice, including the coarse and dense lattices, is a Manhattan set, 
we focus on Manhattan sets that are not lattices, which we call proper. 

4) Video sampling: Let d = 3, let i = 1, 2 be spatial dimensions and let i = 3 be a temporal dimension. 
Whereas video is most commonly sampled with a rectangular lattice, say L(Ai, A 2 , A 3 ), other 
samplings are possible, for example, the Manhattan set M(3, A, k, B) specified by B = { 63 , 03 } 
uses fine spatial sampling every A 3 A 3 seconds and spatial subsampling with factors k\ and A 2 at 
times that are other multiples of A 3 seconds. 
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D. Alternate representations of Manhattan sets 

By the definition of a Manhattan set (11) and Fact 1(a), augmenting an M-collection B by a subset 
b' of some b in B does not change the resulting Manhattan set. Thus many M-collections can generate 
the same Manhattan set. There are, however, unique largest and smallest M-collections that generate any 
given Manhattan set. To find these, we make use of the following. 

Fact 2: Let B and B 1 be M-collections. Then, 

(a) M(B) C M(B') if B C B '. 

(b) M(B) C M(B') if and only if for each b £ B there is b' £ B' such that Lb C L^, or 
equivalently by Fact 1(a), b C b'. 

(c) It is not true that M(B ) = M(B') implies B = B' , or that M(B) C M(B') implies B C B'. 
Proof: 

(a) is obvious. 

(b) If for each b £ B there is b' E B' s.t. Lb C L^, then M(B) = UbgsLb C Ub'gS'Lb' = M(B'). 
Conversely, if M(B ) C M(B'), then for each b £ B, Lb C M(B') = Uf/ e s'Lb', and Fact 1(d) implies 
L h C Lb' for some b £ B . 

(c) Part (a) shows that adding to B some subset of some b € B that is not already in B yields B’ f B 

such that M(B) = M(B'). In this same case, M(B') C M(B), but B' <f B. □ 

The unique largest M-collection that generates M(B) is 

B = {b 7 :b'cb for some b £ L?} , 

which we call the closure of B. Fact 2(a) implies M(B) C M(B) = M(B), and Fact 2(b) implies 
M(B) C M(B). Hence M(B ) = M(B). M(B) is the largest M-collection generating M(B) because 
if M(B') = M(B), and b' £ B', then Fact 2(b) implies there is a b £ B such that Lb' C Lb, and this 
implies b' £ B. Hence, B' C B. 

Removing all elements of an M-collection B that are subsets of another element results in the unique 
smallest M-collection generating M(B), which we denote If. 

E. Manhattan sampling density 

The density of a Manhattan set M(B), i.e., the number of samples per unit area, is obviously less than 
the sum of the densities of the lattices of which it is the union, because each lattice contains all points in 
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Fig. 5. Partitioning of 2D Manhattan grid M({ei,e 2 }) 
with sampling factors fci = /c 2 = 4, which is the union of 
the yellow x’s in V(o,o)> the red A’s in Vn 0 ), and the blue 
□ ’s in V(o, 2 )- The white o’s are in V(i,i), which is disjoint 
from this Manhattan grid. 


Fig. 6. M-partition of Mv for d = 2, ki = 5, &2 = 3. 
Frequency u = 0 lies at the center. Each M-atom A h is 
identified by its b. Note that the cross-shaped Manhattan 
region _M({ei,e 2 }) is also partitioned by M-atoms; in 
particular, M({ei, e 2 }) = A (0 ' 0) U A (1 ’°) u A (0 'b. 


the coarse lattice Lq. Accordingly, we partition the dense lattice L\ in such a way that for any B, M(B) 
is the union of atoms of this partition, and its density can be computed by summing their densities. 

To obtain a suitable partition, let us group into one atom all sites t of L\ having the same answers to 
the following d questions - “Is t % not a multiple of A$&*?” - for i = 1,..., d. Specifically, with a binary 
vector b = (b\,.. ., &<j) indicating the set of i’s for which the answers are “yes”, consider the partition 
{Vb : b C {(). l} f/ }, where the atom corresponding to b is 

Vb = {t : ti is a multiple of Ay A* for i s.t. bi = 0, 

and ti is a multiple of A*, but not Ay A*, for i s.t. 6* = l} 

Fig. 5 illustrates the partitioning of a 2D Manhattan grid. Essentially, it is a partition of the dense lattice 
into collections of cosets of the coarse lattice. 

It is clear from the definition that no t can lie in both Vb and Vb' for b / l/. Hence, the Vb’s 
are disjoint. By comparing the above to the definition (10) of Lb, one sees that Vb' C Lb if and only 
if b' C b. It follows that for any b, Ub'cbHr C Lb- Conversely, if t e Lb, then it is easily seen 
that t E Lb' for b' defined by \( r — 0 for i such that t r is a multiple of A, Ay and b' r = 1 otherwise. 
It follows that Ub'cbVW = Lb, i.e., {Vb} partitions any bi-step lattice, including L±. Moreover, since 
M(B) = UbeB-^b- one can a l so write M(B) = U be ^Vb, i.e., {Vb} partitions any Manhattan grid. 

With this partition in mind, the density of the Manhattan set M(B) is now obtained by summing the 
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densities of each Vb, b G /i. Consider the points of Vo in the fundamental cell 

^k.A = X[0 

1=1 

which has a comer at the origin, lies entirely in the positive hyper-quadrant, and has volume is ru ki\i. 
We see that VoC-Fra is the Cartesian product of d sets A\,..., Ad, where Ai = {A*, 2A, (k t — 1)A,} 
if bj = 1 and A, t = {0} if b- t = 0. Since Vo Cl F^x contains EL-/, =i (E; ~ 1) points, and since the density 
of Vb is this number divided by the volume of F^x, the density of M(B) is 

1 


p(B) = n,-. 6 i= i(fci l) x 

nit h 


nil a, 


( 12 ) 


For example, the densities of several Manhattan sets in three dimensions are given in Table I. 


Manhattan set 

r 

P(F) 

Manhattan lines 

{ei, e 2 , 63 } 

CO 

1 

CO 

Video sampling example 

( e 3 > e 3 } 

(fe 2 + k- l)/k 3 

Manhattan facets 

{ei, ef e^} 

(3k 2 - 3k + l)/k 3 


TABLE I 

Density of several 3-dimensional Manhattan sets with h = k and A, = 1 for all i. 


F. Manhattan partition of frequency space 

As mentioned earlier, our approach to reconstructing an appropriately bandlimited image x(t) from 
M-samples M{B) involves sequentially reconstructing regions of its spectrum 2T(u). Specifically, each 
region will be recovered from the samples in some collection of bi-step lattices contained in the Manhattan 
set. This section describes a partition of frequency space, some of whose atoms will be the reconstructable 
regions. 

Let Aq, denote the Nyquist region for bi-step lattice Lb, i.e., 

Mb = {u : \m\ < — , i = 1,... ,d\, 
l 2 a h ,i J 

with step sizes a?b,i given by (9). For future reference we note that 

Mb' C Mb if and only if b' C b . (13) 
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Since any (d, A, k) Manhattan set is a subset of the dense lattice Li, it follows that the appropriate 
bandlimitation for images reconstructable from any such Manhattan set or any (ci, A, k) lattice is a subset 
of the Nyquist region of the dense lattice, namely, 



Thus, we need only partition Mi. 

Definition 2: The Manhattan partition ( M-partition ) of Mi is {„4 b : b 6 {0, l} rf } where A b is the 
Manhattan atom 6 (M-atom) 

„ 4 b = a b x • • • x a b , 

and a b is the interval, or union of two intervals, 

, A I { Ui ' 2fclA“ — \ Ui \ < 2A“)}> hi = 1 

a i = \ 

^ { u i ■ \ u i\ < 2ki\i }> b-i = 0 

Thus, A h is highpass for all dimensions such that b r = 1 and lowpass for all other dimensions. The 
weight of atom „4 b is ||b||, the weight of b. 

The M-partition is illustrated in Fig. 6 in the case of d = 2, k\ = 5, &2 = 3. We now make several 
easy to deduce, but important, observations. 

1) M-atom A h is the Cartesian product of lowpass intervals (— 2fc 1 A . , 2fc 1 A ) along each dimension such 

that bi = 0, and of the union of two highpass intervals (—— 2I } X ] U [ 2fc 1 A , t^-) along each 

dimension such that bi = 1. Thus, „4 b is the union of 2H b H disjoint orthotopes in M rf . 

2) The M-atoms are disjoint, and their union is Mi. Hence, they comprise a partition of Mi. 

3) The M-atom yl b is a subset of the Nyquist region Mb- Equality holds only for b = 0. 

4) The weight ||b|| of an atom M b is a rough indicator of how highpass or lowpass is the atom. 

5) The lowest weight M-atom, A 0 , is lowpass in all dimensions and equals the Nyquist region Mo of 
the coarse lattice Lo, which is the bi-step lattice with smallest and lowest frequency Nyquist region. 

6) The highest weight M-atom, A 1 , is highpass in all dimensions and contains the highpass “corners” 
of Mi- Its volume is at least large as that of any other atom, and usually larger. We will see later 

Consistent with previous conventions, b is a superscript because it determines a frequency region (an M-atom *4 b ), whereas 
it is a subscript when specifying the Nyquist region A/”b of a bi-step lattice Lb, precisely because it prescribes a sampling. 
Except for b = 0, the M-atoms are highpass regions, whereas Nyquist regions are lowpass. 
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that no proper (d, A, k) Manhattan set permits the reconstruction of these corners since they can 

only be recovered by sampling densely along every dimension, i.e., they arc only recoverable if we 

sample on the dense lattice L±. 

7) If an image x(t) is bandlimited to A f\, then both x(t) and its spectrum X{\\) can be decomposed 
into sums of M-atom components: 

x(t) = y a;b ( t ) 

bG{0,l} d 

*( U )= y xh ( u )> (14 ) 

bG{0,l} d 

where X b (u) = X(u) for u £ A h , A' b (u) = 0 otherwise, and x b (t) is the inverse transform 
of AT b (u). We shall refer to x b (t) and A' b (u) as Manhattan atoms , or simply atoms, of x(t) and 
X(u), respectively. 

It will also be important that the M-atoms can partition the Nyquist region A/"b coiTesponding to any 
bi-step lattice L b, as shown below. 

Fact 3: 

(a) For any b € {0, l} d , the 2H b H M-atoms in {„4 b : b' C b} pai'tition A/b in the sense that 

A/ b = Ub'cb^ b ■ 

(b) b' C b if and only if „4 ,y C Ab- 
Proof: 

(a) Since the elements of {Gl b : b' C b) are disjoint and contained in A/b, it suffices to show 

A/b C [J yl b '. (15) 

b'Cb 

Accordingly, suppose u G A4>. It is then easy to see that u £ A h , where 

y _ I 2k~\l — \ Ui \ < 2A7 
[O’ \ Ui \ — 2k~\l 

which demonstrates (15). 

(b) First, if b' C b, then by (13) and the third observation after the definition of M-atom, A h C 

A/b' C A/b- Conversely, if A h ' C A/b, then by paid (a) A h must be one of the atoms whose union is 

A/b. Hence, b'cb. □ 
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G. Spectral replication induced by bi-step lattice sampling 

For a Manhattan set M(B), the reconstruction algorithm to follow will reconstruct an image x(t) by 
reconstructing its spectrum X(u) one Manhattan atom at a time, in an order to be specified later. In 
particular, for each b e B, it will reconstruct atom X b (u) from the subset of samples corresponding 
to the bi-step lattice Lb, taking into account the aliasing due to previously reconstructed atoms. Using 
the more suggestive s = (si,..., sf), rather than b, to denote a bi-step vector that characterizes a 
sampling, then as reviewed in Section II, sampling with L s replicates the spectrum X(u) at all sites in 
the reciprocal lattice L*. Moreover, if X(u) is bandlimited to A/"i, substituting (14) into (4) gives the 
following decomposition of the sampled spectrum: 

*b(u) = E E * b (u-v). ( 16 ) 

vei* be{o,i} d 

We will refer to the term X b (u — v), and its spectral support A h + v = {u + v : u G -4 b }, as the replica 
of atom X b (u), respectively, A h , at site v. Using this terminology, one sees that reconstructing atom 
X s (u) from the sampled spectrum X s (u) requires accounting for the potential aliasing, i.e., overlap, 
of the replicas of the various atoms of X(u) on X s (u). This requires knowing which replicas of each 
atom will alias, i.e. overlap, A s . More specifically, since the algorithm will only apply to images whose 
spectral support is limited to some subset of the Manhattan atoms, for any pair of bi-step vectors b and 
b', we will need to know whether sampling with bi-step lattice L s causes a replica of atom A h (at some 
site v e L*) to overlap atom X b of the original spectrum. 

Such overlap questions are answered by the following lemma and its corollary. Let i? b denote the 
union of the replicas of all b' atoms induced by sampling with L s . That is, 

flf = U L b ' + 4 ■ 

vei^-o 

Lemma 1: Consider sampling with L s . 

(a) For all b, b' C s, no replica of „4 b overlaps „4 b , i.e., i? b Cl -4 b = 0. 

(b) The replicas of „4 b induced by sampling with L s do not overlap X b if there exists at least one 

dimension i such that Si = 1 and bi / l/-. That is, R^' n -4 b = 0 if (b ® b') A s / 0, where 

b © b' denotes element-wise exclusive or (XOR). 

Proof: 
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(a) If b,b' C s, then Fact 3(b) shows A h ,A h C Af s . Since the sampling theorem for conventional 
rectangular lattice sampling shows that replicas of Af s do not overlap Af s , it follows that the replicas of 
A h cannot overlap A b . 

(b) Let us compare the M-atom 

A h = a b x • • • x a b 

to an arbitrary replica in RY: 

A b + v = (a b + ui) x • • • x (u b + vf ), 

for v € L* — {0}. Note that A b and A b + v are disjoint if and only if a b n (a b + v,) = 0, for some i. 

If, as hypothesized in the lemma, (b©b') As f 0. Then there must exist i such that s, = 1 and either 
bi = 1, b'j = 0 or bi = 0, 6- = 1. First, consider the case that bi = 1, b\ = 0. Then 

^ = {“■ : 2rk sl “ il< 2^} 

and 

(if +Vi = {ui '■ \Ui\ < + Vi . 

Since s t = 1, we have v t = y- for some n r , and one sees from the above that no matter the value of a,., 
(aY + Vi) n a b = 0. Hence, (^ b/ + v) n A h = 0, and so RY Pi „4 b = 0. A similai - argument applies for 
the case that bi = 0, b' t = 1. □ 

The following will provide a key step in showing how to reconstruct appropriately bandlimited images. 

Corollary 1: If ||b / || < ||s||, then replicas in i? b do not overlap A s . 

Proof: We will apply Lemma 1 with b = s. If b' = s, then Part (a) of Lemma 1 shows /( b n A s = 0. If 
b' f s and ||b'|| < ||s||, then there must exist i such that Si = 1 and b) = 0. Therefore, (s 0 b') A s / 0, 
and Paid (b) of Lemma 1 shows R^' fl T s = 0. □ 

H. The multidimensional Manhattan sampling theorem 

Given a Manhattan set M(B ), consider its Manhattan region , which is defined to be the union of the 
Nyquist regions of the bi-step lattices of which it is the union: 

M(B) = U ^ b = U U ^ = U 

be-B be-B b'Cb b'eB 

where the second equality uses Fact 3(a). In this section we show that images bandlimited to Ai(B) 
can be recovered from their M-samples in M (7i); we give an explicit procedure for reconstructing such 
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images from their samples in M(B)\ and we show that the set of such bandlimited images is maximal 
in the Landau sense. 

The key steps are the next two lemmas. The first shows that for any image x(t) whose spectrum 
X(u) is bandlimited to Ai{B), the portion of X(u) in any highest weight M-atom A h can be easily 
recovered from the samples in L b, which are a subset of the M-samples. Specifically, .A b (u) can be 
recovered simply by extracting the A h portion of the sampled spectrum A'b(u) due to sampling with 
Lb- Equivalently, the corresponding component x b (t) of x(t) can be recovered by filtering the sampled 
image Xb(t) with an ideal bandpass filter with frequency support _A b . 

Lemma 2: Suppose M(B) is a Manhattan set and x (t) is an image whose spectrum A"(u) is bandlim¬ 
ited to M.(B). Then if b has maximal weight in B, 

X b (u) = H b ( u)Xb(u), (17) 

where H h ( u) = 1 for u G A b , and 0 otherwise. 

Proof: Consider any b of maximal weight. Since, according to (4), Ab(u) consists of replicas of 
A(u) at the frequencies in L£, since X(\i) can be decomposed into its components on Nyquist atoms 
{„4 b : b' G {0, l} rf }, and since X(u) is bandlimited to JA(B) = Ub'eB"^ b ’ ^ suffices to argue that 
for all b' G B, no replica of A h intersects A h . First, Lemma 1(a) applied with s = b' = b shows that 
no replica of A h with v/0 intersects A h . Second, Corollary 1 and the fact that b has maximal weight 
in B imply that for any other b' G B, no replica of M b can overlap A h . □ 

Once A(u) has been recovered in all such highest weight (highest frequency) M-atoms, the next lemma 
shows that X (u) can then be recovered in the next highest weight M-atoms by canceling the contributions 
due to atoms with larger weight. In effect, the aliasing of one atom comes only from atoms with larger 
weight, i.e., higher frequency. Specifically, for any such b, it shows that A' b (u) can be recovered from the 
spectrum Ab(u) due to sampling with Lb simply by first subtracting each replica X b (u — v), v G L£, 
of every M-atom b' with jb'| > ||b||, and then extracting the M b portion of the resulting “de-aliased” 
spectrum. 

Lemma 3: Suppose M(B) is a Manhattan set and x (t) is an image whose spectrum X{\\) is bandlim¬ 
ited to M(B). If T b (u) is known for all 1/ larger than b, then 

A b (u) = H b ( u) [X b (u) - ^X b '(u)] , (18) 

b': ||b'||>||b|| 

where H b ( u) is defined in the previous lemma, 
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Proof: 

RHS of (18) = H h ( u) [ ^X(u - v) - E E Ab '( u - v )] ( l9 ) 

veL(, b': |jb'||>||b|j veLJ 

= ^ b ( u )[ E E A ' b '( u - v \ 

b': llb'H^llbll vEL£ 

= X b (u), 

where the first equality uses (4), the second uses (14), and the last derives from Lemma 1 and its corollary. 
In particular, for the b' = b term in the above sum. Part (a) of Lemma 1 applied with s = b' = b shows 
that all replicas of _A b (with v f 0) do not overlap A h and, consequently, are eliminated by the filter 
P/ b (u). Corollary 1 implies every replica of M b (with v f 0) does not overlap A h and, consequently, 
is again eliminated by the filter. Since also A h does not overlap A h , the only term in the sum not 
eliminated by the filter is X b (u), which establishes (18). □ 

Note that the sum in (18) can be limited to b' 6 B. Note also that Lemma 3 implies Lemma 2, because 
when b is a largest weight bi-step vector in B, the summation term in (18) is zero, and so (18) reduces 
to (17). 

An alternative way to write (18) is 

X h (u) = H h ( u) [x b (u) - E E A ' b '( u " n ° A.)] (20) 

b': ||b'||>||b|| nECb 

where /3 b = (/? b ,i, • • •, /3 h ,d), with 

|f, if bi = 1 

Pb,i = \ ' 

if } P = 0 

and 

C* b = {n € T, d : n* = 0 for i s.t. bi = 1, and|nj| < ki — 1 for i s.t. bj = 0}. 

To demonstrate (20), we note that since all atoms of the Manhattan partition arc contained in Mi, one 
can eliminate from the last sum in (19) any v such that (Mi + v) Cl Mi = 0- This leads to limiting 
the sum to v such that \v i\<i for each i. Taking into account what v’s are in L' h leads to (20). For 
the usual 2D case, in which B = {(1, 0), (0,1)}, (20) gives a different reconstruction formula than in 
Section III for the spectrum in the coarse Nyquist region, X (: {\i ) = 2f(°’°)(u) (see (5)-(7)). Specifically, 
it subtracts terms involving both 2f^(u) = A' (0 ‘ 1 - j (u) and X H (u) = X 1 ' 1 i l ) (u) from Xc(u), whereas the 
formula in Section III subtracts terms involving X H (u) from Xy(u). Moreover, the summation over n 
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in (20) sums over approximately twice as many values of v. This is because it conservatively includes 
all veL^ such that Mi + v n Mi f 0, whereas the formula in Section III includes only v’s such that 
Mv + v n Me f 0- If desired Cb, in (20) could be replaced by a smaller set C'b.b- that depends on b' 
as well as b. 

The basic idea behind following theorem, which is the main result of this section, is that the process of 
finding X b (u) for smaller and smaller weight b’s can continue until X°, the spectrum in «4°(u) = Mo, 
is found, and all of X(u) is known. As a result, x(t) will also be known. 

Theorem 3: Multidimensional Manhattan Sampling Theorem. Suppose we sample an image x(t) with 
Manhattan set M(B). If the image spectrum X(u) is bandlimited to A4(B ), then for each b e B, X h {u) 
can be exactly recovered from the samples with the following “onion-peeling” approach — apply Lemma 
2 for the largest b’s in B, and then repeatedly apply Lemma 3 for the next largest b’s. Then, x(t) can 
be exactly recovered from 

*(t) = ^ _1 | J] xb ( u )} • 

Proof: From (14) and the bandlimitation of A'(u), it is clear that X(u) can be recovered if 2T b (u) 
is recovered for each b e B. First, A' b (u) can be recovered via Lemma 2 for the largest b’s in If 
which correspond to the highest frequency M-atoms. Next, repeatedly applying Lemma 3 enables one 
to recover A b (u) for the Nyquist atoms corresponding to the largest of the remaining b’s, until X°(u), 
corresponding to the lowpass atom, is recovered. □ 

While this theorem indicates a frequency domain reconstruction, followed by an inverse transform, a 
direct spatial domain reconstruction is also possible. As we now delineate, this involves reconstructing 
each x b (t), b € B, and then using 

*(t) = 

beB 

Taking the inverse transform of (20) yields 

x b (t) = h h (t)-k x b (t) - ^2 Xb (t) 

||b'||>||b|| 

= h h (t)* [iT(b)J]5(t-t / )(a;(t) - J^x b '(t)) 

t'Gl/b 11 b 7 1| ]> || b 11 

= K b ( x ( t7 ) “ 2 ^(t 7 )) h b (t -t'), 

t'GBb ||b'||>||b|| 
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where h h ( t) = J 7_1 {if b (u)}, * denotes convolution, and = fj, 6 = i x Wib This shows 

how a; b (t) can be found — first for the largest b’s from samples of x(t) taken on Lb, then for the next 
largest b’s from samples of x(t) taken on Lb, as well as samples of i: h ' (t) taken on Lb for all larger 
b', and so on. It remains to find a formula for fi b (t). 

To find a formula for /i b (t), which is the inverse transform of // b (u), which in turn has support A b , 
we begin by recalling that A h is the union of 2 b orthotopes in frequency space. Along each dimension 
i, these orthotopes arc centered at zero if b t = 0, and at Lc, if = 1, where 

c . = -( — q_I_^ 

2 V2Aj 2ki\iJ ' 


Additionally, along the ith dimension, these ortho topes have length 

tUi(b) = 


1 1 

bi = 1 

2ki\i ’ 

1 

U \ 1 

bi = 0 


Using these quantities, we can write the filter H h ( u) as 

r d -| r 

H h (u) = n-t(^) * 5(ui) [s(v,i - Ci) + S(ui + Ci) 

-i =1 Wl ^ ^ '-i:b i =0 i:b,=l 

where rect(x) = 1 for \x\ < and 0 otherwise. Note that the first term is an orthotope centered 
and the convolution with delta functions shifts the orthotopes along all dimensions i such that bi 
Taking the inverse transform yields 


at 0, 
= 1. 


d 

fib(f) = J^inj(b)sinc(mj(b)L) • 2cos(27rc*L), 

i —1 i:bi = l 

where sinc(f) = . Observe that, as mentioned in the introduction, these impulse responses depend 

on the ki s and A x ’s, but not the choice of bi-step lattices that comprise the Manhattan set. Moreover, the 
Aj’s have only a simple spatial scaling effect on the filters. 


I. Achievement of Landau lower bound on sampling density 

We now show that the volume of J\4(B), denoted \M(B)\, equals the sampling density of M(B). 
As a result, the set of images bandlimited to M{B) is a maximal set of images that are reconstructable 
from sample set M(B). Equivalently, M(B ) has the smallest density of any sampling set such that all 
images bandlimited to M.(B) arc reconstructable. 
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Since the M-atoms partition we can calculate \M.(B)\ simply by summing over the volumes 

of the M-atoms A h for b G B: 


ix(b)i = ^i^i = E(IKs; 

beB beB 

_ I^bgB Wi-.bi=l^i ~ jO 

rii=l 


1 


2 A j 2kj\j 


n 

i:bi= 0 


kj 


Comparing the above to (12), we see that Ai(B)\ equals the sampling density p(B). 


J. Discrete-space images 

The d-dimensional Manhattan sampling theorem and reconstruction procedures can be straightforwardly 
extended to discrete-space images in d dimensions in the same fashion as for two dimensions. For 
example, for infinite-support images, frequencies need to be scaled by 27 t, and for finite-support images, 
each spatial resolution Tj must be a multiple of kjXj and the Manhattan atoms ,1), need to be redefined 
to be consistent with the DFT, as was done for the discrete Nyquist region A/’ Ql , Q2 . Flere, we simply 
give the main step of the frequency-space onion-peeling reconstruction algorithm for reconstructing a 
Manhattan bandlimited discrete-space image x[t] with finite support sampled with Manhattan set M(B): 

X b [t] = id b [u] DFT-|x b [t] - ]T x b '[t] 

b':||b'||>||b|| 

where x'ft[t] and ./;] ‘ [t] denote, respectively, the Lb subsamplings of the Manhattan samples (scaled by 
L'b), and the previously reconstructed atom x b [t], and iF b [u] denotes an ideal bandpass filter for atom 
Alb- 



V. Concluding Remarks 

In two dimensions, this paper has shown that from samples of a Manhattan set one can perfectly 
reconstruct any image that is bandlimited to the union of the Nyquist regions of the horizontal and 
vertical rectangular lattices comprising the Manhattan set. It also prescribed a straightforward linear 
reconstruction procedure, for continuous- and discrete-space images. 

For three and higher dimensions, this paper has identified Manhattan sets as the union of a finite 
number of bi-step rectangular lattices, with the result that many Manhattan geometries are possible. It 
introduced an efficient binary-vector representation of bi-step lattices, and consequently Manhattan sets, 
which enabled the specification of a partition of the dense rectangular lattice into collections of cosets of 
the coarse lattice. This, in turn, enabled the density of a Manhattan to be computed. The representation 
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of bi-step lattices also enabled a partition of the Nyquist region of the dense rectangular lattice, which 
in turn enabled a precise analysis of the aliasing, i.e., spectral overlaps, by the atoms of any particular 
type in the spectral replicas induced by any particular bi-step lattice subsampling. With this, it was 
shown that images bandlimited to the union of the Nyquist regions of the bi-step lattices comprising 
the Manhattan set can be perfectly reconstructed using an an efficient closed-form onion-peeling type 
reconstruction algorithm that reconstructs the image spectrum working from higher to lower frequency 
atoms of the partition. At each step, the algorithm works with samples of one particular' bi-step lattice (of 
the Manhattan set), and obtains the spectrum of the image in the corresponding atom of the frequency 
partition by subtracting contributions due to aliasing of previously determined atoms of the partition. 
Both frequency- and time-domain versions of the algorithm were given. It was also shown that the set 
of Manhattan bandlimited images is maximal in the Landau sense. To the best of our knowledge, this is 
the first demonstration that images bandlimited to the union of Nyquist regions can be recovered from 
the union of the corresponding lattices. 

There are several avenues for future research. One could seek to extend the results to continuous-space 
images whose spectra contain delta functions, e.g. periodic images. Second, instead of a recursive onion¬ 
peeling reconstruction, one could seek direct closed-form linear reconstructions, as in [20], which might 
be useful for implementations, though they might have less intuitive appeal. This is not difficult in 2D, 
but is more challenging in higher dimensions. Finally, whereas M-sampling can be viewed as sampling 
(in various ways) on the boundaries of a rectangular (hyper-rectangular) lattice tessellation, one could 
seek sampling theorems and reconstruction procedures for images sampled on the boundaries of other 
lattice tessellations, such as a hexagonal tessellation. 
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